Papers with lightweight adapters
A Lightweight Mixture-of-Experts Neural Machine Translation Model with Stage-wise Training Strategy (2024.findings-naacl)
Copied to clipboard
| Challenge: | Using mixture-of-experts (MoE) to deal with language heterogeneity is a challenge in neural machine translation (NMT). |
| Approach: | They propose a lightweight MoE-based NMT model that is trained via an elaborate stage-wise training strategy. |
| Outcome: | The proposed model achieves stable improvements in translation tasks by introducing fewer extra parameters compared to baseline models. |
Mixture-of-Partitions: Infusing Large Biomedical Knowledge Graphs into BERT (2021.emnlp-main)
Copied to clipboard
| Challenge: | Infusing factual knowledge into pre-trained models is fundamental for many knowledge-intensive tasks. |
| Approach: | They propose an infusion approach that partitions a large knowledge graph into smaller sub-graphs and infuses their specific knowledge into various BERT models using lightweight adapters. |
| Outcome: | The proposed approach improves the underlying BERTs and achieves new SOTA performance on six downstream tasks. |
Video-Skill-CoT: Skill-based Chain-of-Thoughts for Domain-Adaptive Video Reasoning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for chain-of-thought reasoning fail to adapt to domain-specific skills over video content. |
| Approach: | They propose a framework that automatically constructs and leverages skill-aware CoT supervisions for domain-adaptive video reasoning. |
| Outcome: | The proposed framework outperforms strong baselines on three video understanding benchmarks. |